A Novel Stochastic Decoding of LDPC Codes with Quantitative Guarantees

Low-density parity-check codes, a class of capacity-approaching linear codes, are particularly recognized for their efficient decoding scheme. The decoding scheme, known as the sum-product, is an iterative algorithm consisting of passing messages between variable and check nodes of the factor graph. The sum-product algorithm is fully parallelizable, owing to the fact that all messages can be update concurrently. However, since it requires extensive number of highly interconnected wires, the fully-parallel implementation of the sum-product on chips is exceedingly challenging. Stochastic decoding algorithms, which exchange binary messages, are of great interest for mitigating this challenge and have been the focus of extensive research over the past decade. They significantly reduce the required wiring and computational complexity of the message-passing algorithm. Even though stochastic decoders have been shown extremely effective in practice, the theoretical aspect and understanding of such algorithms remains limited at large. Our main objective in this paper is to address this issue. We first propose a novel algorithm referred to as the Markov based stochastic decoding. Then, we provide concrete quantitative guarantees on its performance for tree-structured as well as general factor graphs. More specifically, we provide upper-bounds on the first and second moments of the error, illustrating that the proposed algorithm is an asymptotically consistent estimate of the sum-product algorithm. We also validate our theoretical predictions with experimental results, showing we achieve comparable performance to other practical stochastic decoders.Comment: This paper has been submitted to IEEE Transactions on Information Theory on May 24th 201

Stochastic belief propagation: Low-complexity message-passing with guarantees

The sum-product or belief propagation (BP) algorithm is widely used to compute exact or approximate marginals in graphical models. However, for graphical models with continuous or high-dimensional discrete states and/or high degree factors, it can be computationally expensive to update messages. We propose the stochastic belief propagation algorithm (SBP) as a low-complexity alternative. It is a randomized variant of BP that passes only stochastically chosen information at each round, thereby reducing the complexity per iteration by an order of magnitude. We prove that it enjoys a number of rigorous convergence guarantees: for any tree-structured graph, the SBP updates converge almost surely to the BP fixed point, and we provide non-asymptotic bounds on the mean absolute error. For general graphs that satisfy a standard contraction condition, we establish almost sure convergence to the unique BP fixed point, as well as non-asymptotic guarantees on the mean squared error, showing that it decays as 1/t with the number of iterations t. We also provide high probability bounds on the actual error